ORCA-SPOT: An Automatic Killer Whale Sound Detection Toolkit Using Deep Learning

Large bioacoustic archives of wild animals are an important source to identify reappearing communication patterns, which can then be related to recurring behavioral patterns to advance the current understanding of intra-specific communication of non-human animals. A main challenge remains that most large-scale bioacoustic archives contain only a small percentage of animal vocalizations and a large amount of environmental noise, which makes it extremely difficult to manually retrieve sufficient vocalizations for further analysis – particularly important for species with advanced social systems and complex vocalizations. In this study deep neural networks were trained on 11,509 killer whale (Orcinus orca) signals and 34,848 noise segments. The resulting toolkit ORCA-SPOT was tested on a large-scale bioacoustic repository – the Orchive – comprising roughly 19,000 hours of killer whale underwater recordings. An automated segmentation of the entire Orchive recordings (about 2.2 years) took approximately 8 days. It achieved a time-based precision or positive-predictive-value (PPV) of 93.2% and an area-under-the-curve (AUC) of 0.9523. This approach enables an automated annotation procedure of large bioacoustics databases to extract killer whale sounds, which are essential for subsequent identification of significant communication patterns. The code will be publicly available in October 2019 to support the application of deep learning to bioaoucstic research. ORCA-SPOT can be adapted to other animal species

Simulated movies of fluorescently stained bacteria

Wiesmann V, Bergler M, Münzenmayer C, Wittenberg T. Simulated movies of fluorescently stained bacteria. Fraunhofer Institute for Integrated Circuits, Erlangen, Germany; 2017.Simulated biomovies of fluorescent cells were created by employing cell simulation based on shape, texture, growth and motion information. They are depicted in fluorescent channels where noise and other artifacts were additionally modeled. While the original cell simulation software (Wiesmann et al. 2013) has been extended for biomovie simulation, the step for bacterial growth simulation is similar to image simulation. First, the cell shape is calculated. Second, the cell position on the image grid is calculated. Third, the cell texture is added. Fourth and last, imaging artifacts and noise are added to the final image

EXACT: a collaboration toolset for algorithm-aided annotation of images with annotation version control

In many research areas, scientific progress is accelerated by multidisciplinary access to image data and their interdisciplinary annotation. However, keeping track of these annotations to ensure a high-quality multi-purpose data set is a challenging and labour intensive task. We developed the open-source online platform EXACT (EXpert Algorithm Collaboration Tool) that enables the collaborative interdisciplinary analysis of images from different domains online and offline. EXACT supports multi-gigapixel medical whole slide images as well as image series with thousands of images. The software utilises a flexible plugin system that can be adapted to diverse applications such as counting mitotic figures with a screening mode, finding false annotations on a novel validation view, or using the latest deep learning image analysis technologies. This is combined with a version control system which makes it possible to keep track of changes in the data sets and, for example, to link the results of deep learning experiments to specific data set versions. EXACT is freely available and has already been successfully applied to a broad range of annotation tasks, including highly diverse applications like deep learning supported cytology scoring, interdisciplinary multi-centre whole slide image tumour annotation, and highly specialised whale sound spectroscopy clustering

Open-source workflow approaches to passive acoustic monitoring of bats

The work was funded by grants to PTM from Carlsberg Semper Ardens Research Projects and the Independent Research Fund Denmark.The affordability, storage and power capacity of compact modern recording hardware have evolved passive acoustic monitoring (PAM) of animals and soundscapes into a non-invasive, cost-effective tool for research and ecological management particularly useful for bats and toothed whales that orient and forage using ultrasonic echolocation. The use of PAM at large scales hinges on effective automated detectors and species classifiers which, combined with distance sampling approaches, have enabled species abundance estimation of toothed whales. But standardized, user-friendly and open access automated detection and classification workflows are in demand for this key conservation metric to be realized for bats. We used the PAMGuard toolbox including its new deep learning classification module to test the performance of four open-source workflows for automated analyses of acoustic datasets from bats. Each workflow used a different initial detection algorithm followed by the same deep learning classification algorithm and was evaluated against the performance of an expert manual analyst. Workflow performance depended strongly on the signal-to-noise ratio and detection algorithm used: the full deep learning workflow had the best classification accuracy (≤67%) but was computationally too slow for practical large-scale bat PAM. Workflows using PAMGuard's detection module or triggers onboard an SM4BAT or AudioMoth accurately classified up to 47%, 59% and 34%, respectively, of calls to species. Not all workflows included noise sampling critical to estimating changes in detection probability over time, a vital parameter for abundance estimation. The workflow using PAMGuard's detection module was 40 times faster than the full deep learning workflow and missed as few calls (recall for both ~0.6), thus balancing computational speed and performance. We show that complete acoustic detection and classification workflows for bat PAM data can be efficiently automated using open-source software such as PAMGuard and exemplify how detection choices, whether pre- or post-deployment, hardware or software-driven, affect the performance of deep learning classification and the downstream ecological information that can be extracted from acoustic recordings. In particular, understanding and quantifying detection/classification accuracy and the probability of detection are key to avoid introducing biases that may ultimately affect the quality of data for ecological management.Publisher PDFPeer reviewe

ORCA-SPY enables killer whale sound source simulation, detection, classification and localization using an integrated deep learning-based segmentation

Acoustic identification of vocalizing individuals opens up new and deeper insights into animal communications, such as individual-/group-specific dialects, turn-taking events, and dialogs. However, establishing an association between an individual animal and its emitted signal is usually non-trivial, especially for animals underwater. Consequently, a collection of marine species-, array-, and position-specific ground truth localization data is extremely challenging, which strongly limits possibilities to evaluate localization methods beforehand or at all. This study presents ORCA-SPY, a fully-automated sound source simulation, classification and localization framework for passive killer whale (Orcinus orca) acoustic monitoring that is embedded into PAMGuard, a widely used bioacoustic software toolkit. ORCA-SPY enables array- and position-specific multichannel audio stream generation to simulate real-world ground truth killer whale localization data and provides a hybrid sound source identification approach integrating ANIMAL-SPOT, a state-of-the-art deep learning-based orca detection network, followed by downstream Time-Difference-Of-Arrival localization. ORCA-SPY was evaluated on simulated multichannel underwater audio streams including various killer whale vocalization events within a large-scale experimental setup benefiting from previous real-world fieldwork experience. Across all 58,320 embedded vocalizing killer whale events, subject to various hydrophone array geometries, call types, distances, and noise conditions responsible for a signal-to-noise ratio varying from −14.2 dB to 3 dB, a detection rate of 94.0 % was achieved with an average localization error of 7.01∘. ORCA-SPY was field-tested on Lake Stechlin in Brandenburg Germany under laboratory conditions with a focus on localization. During the field test, 3889 localization events were observed with an average error of 29.19∘ and a median error of 17.54∘. ORCA-SPY was deployed successfully during the DeepAL fieldwork 2022 expedition (DLFW22) in Northern British Columbia, with a mean average error of 20.01∘ and a median error of 11.01∘ across 503 localization events. ORCA-SPY is an open-source and publicly available software framework, which can be adapted to various recording conditions as well as animal species

Face mask recognition from audio: the MASC database and an overview on the mask challenge

The sudden outbreak of COVID-19 has resulted in tough challenges for the field of biometrics due to its spread via physical contact, and the regulations of wearing face masks. Given these constraints, voice biometrics can offer a suitable contact-less biometric solution; they can benefit from models that classify whether a speaker is wearing a mask or not. This article reviews the Mask Sub-Challenge (MSC) of the INTERSPEECH 2020 COMputational PARalinguistics challengE (ComParE), which focused on the following classification task: Given an audio chunk of a speaker, classify whether the speaker is wearing a mask or not. First, we report the collection of the Mask Augsburg Speech Corpus (MASC) and the baseline approaches used to solve the problem, achieving a performance of [Formula: see text] Unweighted Average Recall (UAR). We then summarise the methodologies explored in the submitted and accepted papers that mainly used two common patterns: (i) phonetic-based audio features, or (ii) spectrogram representations of audio combined with Convolutional Neural Networks (CNNs) typically used in image processing. Most approaches enhance their models by adapting ensembles of different models and attempting to increase the size of the training data using various techniques. We review and discuss the results of the participants of this sub-challenge, where the winner scored a UAR of [Formula: see text]. Moreover, we present the results of fusing the approaches, leading to a UAR of [Formula: see text]. Finally, we present a smartphone app that can be used as a proof of concept demonstration to detect in real-time whether users are wearing a face mask; we also benchmark the run-time of the best models

The ACM Multimedia 2022 Computational Paralinguistics Challenge: vocalisations, stuttering, activity, & mosquitoes

The ACM Multimedia 2022 Computational Paralinguistics Challenge addresses four different problems for the first time in a research competition under well-defined conditions: In the Vocalisations and Stuttering Sub-Challenges, a classification on human non-verbal vocalisations and speech has to be made; the Activity Sub-Challenge aims at beyond-audio human activity recognition from smartwatch sensor data; and in the Mosquitoes Sub-Challenge, mosquitoes need to be detected. We describe the Sub-Challenges, baseline feature extraction, and classifiers based on the 'usual' ComParE and BoAW features, the auDeep toolkit, and deep feature extraction from pre-trained CNNs using the DeepSpectrum toolkit; in addition, we add end-to-end sequential modelling, and a log-mel-128-BNN

A New CYP2E1 Inhibitor, 12-Imidazolyl-1-dodecanol, Represents a Potential Treatment for Hepatocellular Carcinoma

Cytochrome P450 2E1 (CYP2E1) is a key target protein in the development of alcoholic and nonalcoholic fatty liver disease (FLD). The pathophysiological correlate is the massive production of reactive oxygen species. The role of CYP2E1 in the development of hepatocellular carcinoma (HCC), the final complication of FLD, remains controversial. Specifically, CYP2E1 has not yet been defined as a molecular target for HCC therapy. In addition, a CYP2E1-specific drug has not been developed. We have already shown that our newly developed CYP2E1 inhibitor 12-imidazolyl-1-dodecanol (I-ol) was therapeutically effective against alcoholic and nonalcoholic steatohepatitis. In this study, we investigated the effect of I-ol on HCC tumorigenesis and whether I-ol could serve as a possible treatment option for terminal-stage FLD. I-ol exerted a very highly significant antitumour effect against hepatocellular HepG2 cells. Cell viability was reduced in a dose-dependent manner, with only the highest doses causing a cytotoxic effect associated with caspase 3/7 activation. Comparable results were obtained for the model colorectal adenocarcinoma cell line, DLD-1, whose tumorigenesis is also associated with CYP2E1. Transcriptome analyses showed a clear effect of I-ol on apoptosis and cell-cycle regulation, with the increased expression of p27Kip1 being particularly noticeable. These observations were confirmed at the protein level for HepG2 and DLD-1 cells grafted on a chorioallantoic membrane. Cell-cycle analysis showed a complete loss of proliferating cells with a simultaneous increase in S-phase arrest beginning at a threshold dose of 30 μM. I-ol also reduced xenograft tumour growth in nude mice. This antitumour effect was not associated with tumour cachexia. I-ol was not toxic to healthy tissues or organs. This study demonstrates for the first time the therapeutic effect of the specific CYP2E1 inhibitor I-ol on the tumorigenesis of HCC. Our findings imply that I-ol can potentially be applied therapeutically on patients at the final stage of FLD. © 2021 Torsten Diesinger et al